Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction

Stable concurrent learning and control of dynamical systems is the subject adaptive control. Despite being an established field with many practical applications a rich theory, much development in for nonlinear revolves around few key algorithms. By exploiting strong connections between classical techniques recent progress optimization machine learning, we show that there exists considerable untapped potential algorithm both dynamics prediction. We begin by introducing first-order adaptation laws inspired natural gradient descent mirror descent. prove when are multiple consistent data, these non-Euclidean implicitly regularize learned model. Local geometry imposed during thus may be used to select parameter vectors—out will achieve perfect tracking or prediction—for desired properties such as sparsity. apply this result regularized predictor observer design, concrete examples, consider Hamiltonian systems, Lagrangian recurrent neural networks. subsequently develop variational formalism based on Bregman Lagrangian. its Euler Lagrange equations lead descent-like momentum, recover their analogues infinite friction limit. illustrate our analyses simulations demonstrating theoretical results.